In TV systems, the first stage of video enhancement generally is noise reduction in order to obtain noise-free video sequences. Various noise reduction methods have been developed but few of them are used in real products due to the artifacts. Most of the noise reduction methods can be classified into three categories: (1) spatial (2D) noise reduction, (2) temporal noise reduction, and (3) 3D noise reduction. The 3D noise reduction is a combination of 2D and temporal noise reduction.
Spatial noise reduction applies a filter of a small local window on every pixel of the current video frame. Such a filter usually is regarded as a convolution filter based on a kernel. The most common filters are the mean filter, the Gaussian filter, the median filter, the sigma filter, etc. Mean filtering is the simplest method of smoothing images and reducing noise by taking the mean of a small local window as the filtered result. Generally, a 3×3 square kernel is used, and as such it is simple to implement. Mean filtering, however, causes severe blurring of images.
Gaussian filtering uses a “bell-shaped” kernel to remove noise. Gaussian filtering equivalently is a weighted average operation of the pixels in a small local window. However, Gaussian filtering also introduces blurring, but the severeness can be controlled by the standard deviation of the Gaussian.
Median filtering is a nonlinear method that sorts the pixels in a small local window and takes the median as the filtered result. Median filtering does not create new unrealistic pixel values, and preserves sharp edges. Also, an aliasing pixel value does not affect the filtered result. However, as the number of input pixels increases, the computational cost of sorting is also increases, making it costly to implement practically.
To address the problem of blurring, some edge-oriented spatial filtering algorithms have been developed. Such algorithms, however, require expensive hardware and introduce artifacts when edge-detection fails, especially in very noisy images. Other algorithms convert images into frequency domain and reduce the high frequency components. Since image details are also high frequency components, such methods also blur the image.
Temporal noise reduction first examines motion information among the current video frame and its neighboring frames. It classifies pixels into motion regions and non-motion regions. In a non-motion region, a temporal filter is applied to the pixels in the current frame and its neighboring frames along the temporal axis. In a motion region, the temporal filter is switched off to avoid motion blurring. Generally, temporal noise reduction is better in keeping the details and preserving edges than spatial noise reduction.
One such conventional temporal noise reduction method is applied to two frames: (1) one frame is the current input noisy frame, and (2) the other frame is the previous filtered frame. Once the current frame is filtered, it is saved into memory for filtering the next incoming frame. Motion and non-motion regions between the two frames are examined. In a non-motion region, pixels are filtered along the temporal axis based on the Maximum Likelihood Estimation, outputting high quality filtered images, but with a drawback of unevenness, caused by switching off temporal filtering in motion regions. For example, some unfiltered regions close to the moving objects appear as noisy tailing. To overcome this problem, a bidirectional temporal noise reduction has been utilized, which filters the motion region in either forward direction or backward direction to remove the tailing effects. However, such a method requires more frame buffer and frame delay. Further, such a method cannot perform temporal noise reduction on a moving object.